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ABSTRACr 



Lord* a Paradox ia analyzad .£n terms of a simple matheinatical 
• model 'for causal Inference, The resolution of Lord's Paradox from 

this perspective has two aspects , « Firsts the descriptive r non-causal 
, /= cpnclusions of the two hypdthetical statisticians are both correct. 

Th^ appear contradictory only because they ' de quite differr'^n 

aspects the data/ Second^ the causal inferences of ) the s tat l^r - 
; clans are nelther^correet nor incorrect since they are Msed on 

"different assumptions that our inathematical modej. makes explicit but 
' neither assumption can be tested using the. data set that is described 
. In the •exainple. ^We identify these differing assumptions and show how 

each may be used- to j us tify^ the differing causal conclusions of the . 

two statisticians . In addition to ■'analyiging the classic "diet" 

# ' . : ' ^ ...... 

f example which Lord used to ^introduce his paradox^ we also examine 
three other eKamples that appear in the three papers Vhere Lord dis- 
.cusses= the paradox and related masters. . 
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L. Introducbion \ ' ' * 

■ ■ • ■ I- . , ... ■ • ■ ■ " . ; ■■ ■ - 

Lord's ParadoK firpt appeared in a short , tworpage article . CLord>_, 

1967). in Psychological Bullatin . This article presents a remarkable 

" contrast between two statisticians whtf^-.draw widfely different conclusiona 

from the same set ' of data* Tfie culprit appears to be that the analysis 

of covariance cannot be counted on to make a proper allowance for^ , 

uncontrolled preeKisting differences between natural grpups , Much to 

, the dismay o£ the editor of Psychological Bulietin p Lord did not resolve 

his paradox. This fact increases the interest In the questions it -^^^ 

.raises. The impact of the paper has been an eKtenslve analysis and 

criticism of the use of the analysis of covarianc^ that still continues 

= . . ^ ■ . f ' a 

(e.g.p' Games, 1976 and Lindley and JJovick, 1981). .Lord wrote t^o 

' . ' . " . ' ' ■ - ' y , : . ■ ' \ \.- 

, additional short pieces on the paradox " a second article in Psych olo- 
gical Bulletin , <Lord^^ 1^68) and , an entry in the Encyclopedia of Educa- 
tional Evaluation (Lord, 1973 in Anderson, et al * , 19 73* page 233). We 
basci our dlscuBsions on these three articles by Lord* 

Lord^ uses eKample's t^3 illustrate his points, and there are four 
examples'' discussed in the three papers* ■ Our approach d^^ffers from 
Lord's in th^t we first present a mathematical framework that is com- 
plex enough to accoimnodate what: we regard to be the' 'important features 
of the examples treated by Lord, and we then apply this framework to 
each of his examples. As will become evident, we belieVG that there 
are several different issues that ayise i^i^these exatoples, and we feel - 
thatf our ihathematical framework provides the-structure for a precise 
analysis * . ' ' 



Our paper is organized as followsi In Section 2 we describs the 
^ ' * . / ■ ^ " - * » ■ = . ■ : 

,jeneral raathematical framework -or model for causal inference. In ^ 

Section 3;we apply this ganefal framework to each of thd example? 
appearing in Lord's three papers. Section 4 gives our general cdnclu^ 
sipnC regarding; the nature of Lord* s ParadoK* We^^ include an ^^Appendix . 
whloh indicates various' related results >that follQW from our'modei. 
2. A Mode 1 f or^iGA^usal Irt'ferenae ; ' . ' ' ; ' 

In this section we"" describe our model for causal infarenae and- ; 
derive the results from It that 4^fe heed for the examples that Uord J* 
discusse*d* Mori technical consequences of the .model are derived in- 

'h- ■ ■ ^ .;^. " . : ■' 

\ ^ : ^ ■ l ' 

the Appendix. ^ ^ * i ^ 

' . * ' * ■ ^ ^ ■ ' ' . ^ ^ ' : ' 

2*1 The Elements of the Model . ' / / ^ . 

The chief-' isfeue *that ^ is of concerB-Jh Lord' s Paradox is the 
attribution of cause* Much. has been written about causation but our 
point of departure is the analysis of causal effects given in Rubin 
(1974, 1977, 1978, 1980>* However,^ it will be sufficient for our pur-- 
poses to deal with' a simplified j £Opulati'dn=le of Rubin's 

model* We have used this ^simpii.fd#d model :^ elsewhere (Hoiland and 
Rubin, l^So) to analyze, causal inference in retrospective^i^case^contro^l 
studies of ten/ used in^medlcal research.^ = >:*^ \ ^ / . ; 

Our model| is similar to those used to describe many simple statis- 
tical problema. However, we are absolutely explicit about certa:in dis- 
tinctions- an^ elements that are usually iejt. Implicit in other discus- 
sions* We believe that it. is impossible t% give a coherent analysis , 
of causal inference without/being at least asjexpliclt as we are- here* 

the basic elements of^ ou^'model arei / ' 



; a 'population of uh^ts, if 5 , = . , \ 

, (b) an "experimental inanlpulatlon'% with levels fe or c, _ 
' I and Its associated indicator variables S, ' 

(c) .a aubpopulation indicator variable^ G, 

(d) ' an outcome variable 5 and.* * ' 

^ (e) an concomltaftt variable^ 32# ■ - v " 

Each of these cotnponenta to the model needs further specif icatidrt and 

we Jdo/this /in the neKt subsection; Figure 1 suimnarizee this framework^ 
^ ■■ - ■ ■ ■ ■ ^ 

.V "^ ■ - • - ^ " '.^ _ , ^ . ^ 

^ ' _ . Figure 1 about here 

2 , 2^ Discussion of the Elements ' of the Model a 

The population P of units underlies the rest of the model, Typi^ 
■ dal eSamples of *'units" are human sub j ects ^ or ratSj'or households , or 
Vcornrseeds* All variables ^ are aspumed to be functions that are defined 
on^' every unit in P*^ All probabilities j distributions and expected i 
values are computed^ ver f, A probability will mean nothing more rior 
1. ^■■:s than a proportion of^ units in P. The expected value of a variable 
merely its average value ov^er all of P. Conditional expected values 
' ire subgroup averages where the subgroups are' defined by the condition^ 

Ing statement. In Figure 1 there are N units in P* ^ = 

'The "eKperimental inanlpulation'^ is the focus of all causal infer- 
ence in our model. It is important to realise that by using the term 
' experimental manipulation" we do not mean to limit our discussion to 
the activities within a controlled randomiied laboratory study. We do 
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Figure 1: A Frajnswork for Causal inference - 
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mean tb include any .sort of weli^daflnsd eKperierice to which ^ach of 
. the units in P ma}?; or may not be eKpoaed, The key* notion^ is the poten^. 
tial for eKposing each unit to any one of/ the eKperimental conditions 
in the s tudy> For causaj, inference, it is ' critical that each unit be \ 
^potentially expo sable to any one of the experimental conditions .As an 
aKample^ the schooling- a student receives is an experimental manlpula^ 
tlfan in our sense* whereas the student's race or gender Is not."' 

For simplicity,, In. this paper we shall assume that ther^ are just 
two different experimental conditions or leveis; of treatment* denoted t:^ 
(treatment) and c (control). We will let S be a variable that^ indid'ates 

' y . . ^ ' ^ / I. ^ ^ ^ 

the experiinental^conditlon to which each unit in P is exposedV that isj 
S^^ t indicates the unit is exposed to t* while S ^ c indicates exposure 
to c* In a controlled study, S*is constructed by the eKperimenter , . In 
an uncontrolled study S is, determined to some extent by factors beyond 

the experim^terVs control.-^ In either case^* the critical feature of - 

the ^*experlmentai inanipulation" Is that the' valui of S for each unit -. 
could have been -different* , ' ; 

/ We will make the simplifying assumption that S is defined on all ^ 
of P /go that for- each unit either S t or S ^ c. In two of the eKamples 
in Section 3, S is a constant over P Cl,e.^ there is only one treatment 
to which units are actually exposed)* In one example^ there is no S ^ ^ 
iince. there is^ no identified treatment* Our model is at the "popula-^ 
tion level" . because we do not consider the inference problems associated 
with the sampling of units in P for study/ The model described by. 
Rubin Cl974^ U??, 3-978, 1980) deals with the added complexity of the 
sampling of units* ' > , • ^ X 
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■ The "term **lndependent^' variable" is ten .used to refer to a treat 
inent indicator .variable like but i^ iia also applied more loosely "to 
include an entirely diff arent type of variable .^^ In" our model this 
. second type of variable is the sub^opulatioh ihdxcaM^yariable ^# 
Evans arid Anas tasip / (1968)^ aAdng otherSp distinguish clearly fee tween 

genuine independent variables^ tr%atinenta Chat can be .manlpulafiad" 

k ' ■ ' ■ ^ * i ' . % / 

and "elasalf Icationa or other variables which- descrlba. the intact 

groups ^ Lord is gMite aware of the dlptinctipn and,.in facts das- 

■ ' ^ * ■ -J ■ " -. ' * ■ . . . ' . 1 = ' 

cribes his paradox as_"a problem that "arises in interpreting data on' 

^ - : . . • / " " . r V * ^ ^ ' \. . ^ V 

. preexisting groups,'' He also ref ere to the impossibility of^ random 
assignment in the "comparison of the feducatibnal achievements of^ 
different racial groups," In our model we have twq^ifferent variabie! 
S and in order to represent both cases^ The '0^ari|ab4e G indicates 
the subpopulation membership of each units such as race or gender of /' 



students, "yarleties" of coruj etc* Unlike "Sj it is not possible for^ 
* the value of for each unit to have been other than what it is, "For 

the purposes of this paper,, we have a si'ngle subpopulatidnJindicator 
, variable G which has only fiwo possible values (G ^ 1,2)^ , indicating,^ 
for example, male and female studentSs as shown in Figure 1. 

By the "outcome variable*' Yj .we mean to convey the usual notion 
of dependent or criterion variable, with one important extension. 
When* there is an experimental manipulation, there are multiple yer= 
sions ^f Yj one for each treatment condition. In our case, these are 
denoted by Y^ and Y^i The interpretation of thesa^ twa yalues of ^ for 
p a given unit is that Y^ is the value of Y ttt^^' w'ould be observed if .* 
the unit were Exposed to t while Y is th^' valiJi of Y that would be 



1^ 



observed' 'Qn . tha same unip if it were axposed to c. The basic notion 
that a treatment Inf luenias the dependent variabla is - f orniali?ad in- ^ 



■■ the model by the two values ^ Y. and Y If t Inf luencas Y, ttien the 
pffecit of the aKperimental\manipulation Is to make; the .value 
diffardnt from' the vaiua Y^^^ for eachV unit. The null hypothesis of : . 
■ilno^^ treatmant-af£ac£!! jCIilA^ a trpngM t jprm) cMMSponds tq / 
fSr allTunlfcs Vih P* eKperimental marii- 
pulations there is only , one version of Y, In such leases, jre d^^ 
put a.subscript on Y, nor dd we subscript Y when we are referring to 
it without reference to th%J treatment conditions. • ^ - 

Central' to Lord's faradoK is rtie availability of a variable X : 
that is auxilikry to the. outcome variable' Y," Wa will eall^^X a: ■ ^ V^l' " 
:cQncomitant variable to distinguish. it fram Y* However, in. Lord's 
eKaiplea t^ere are two distinct types . of concomitant variables that 
?:arise — those that aye and those th^t are /not potentially inf luenced ^ 
by the eKperimentai manipulation*/- This sfatev^ of . affairs can -be exV 
pressed^ as fidildws T ^ Let X" arid X denote the' val^e of X that would be" ~ 
observed if the unit were exposed to .t or to c^ respectively If. 
X ^X for all units ^ then X is not .influenced by the treatment,, and; , 
in this special case we sh^ll tall X a covariate ; , If ^ X^ for some 
units, than It is not a covariate, but wa will :s till i^e the laora general 
term^" concpmitant variabla to describe X in this-- case* By definitioni \ 
the -subpopuiation indicator ^variable G is an eKample^ of a covariat'e,. In. 
real— life research designs," the question of whether or not X - ^X > can 
be quite serious and difficult to answer, / Under the usual, qircunistances 
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of educational rasearch^ pratests are covariatas bacsuee they ;1are — 
racorded prior to'^thp exposuM of units to the treatment eondit ions? an^ 
they aifa,. hanca^ not aff acted by axposura to . ont treatment ar ano ther . ; .- 

2v,3 Thraa Kinds of. Studlas ' . /: . . ^ ^ '^^ - - 

•- . - ' , ' ' - •• = ■ - = -. ■ ^ ■ -. -' t ■ . 

The primary purposa of. model ia to allow an explicit, descrip- ^ 
tion of tha^quaTttlt±es_that_^risa In three type^ of studies that ^ W^^^^ 
shall refer to as* *. 

. A) Dascriptiva studieSj _ , _ _ _ : 

» B)~ UncQn^rollad causi.l^st'udlaSj^arid . . . 

- C) Controlled causal S:tudies* ' = ^ - . . 

Although this as - an p^arsimpliiied categorization of research s^tudias , 
we baliava that it captures unp or tan t distinctions that are. germane, to" 
our analysis of Lord '^s ParadDK* t v ■ 

. 'A descriptive ^tudy has no. experimental' manipulation so that there 
Is only one version of Y and of X- and 'no treatment indicator variable S * 
-Controlled and- uncont^ollad causal studies both have an experiment 
tai manipulktion and differ only in. the degree of control that tfije 
eicparimenterj has over, the treatment iridicatorv '^p In a ■controlled 
causal study, th6 values of S are determined by the eKperitnentar and can 
depend on numarous aspects of each unit , e*g* ^ subpopulat ion member shlp% 
values of covariateSs' but not on "the value of . Y . or since the value 
of the out coma Variable is observed after of S ar^e determined 

by the expisrimente^* tn an unconttolled causal study: the values of S 
are deteroined by facfcors that are beyond the axperimenter *s^ control * - 
Of critical .top^ortanca is the fa^t that, in a controlled, caiusal study^ 
S can be made to be statistically indapendent of Y -and Y whereas in in 



uncontroMed ^causal study ^his is not true. All o£ Lord- s examp las / 
concern either deacriptive studies *or uncontrolled causa^l studies i . 
these are the types of studies that comaonly arise in the behavioral v 

sciences and involve "preexisting groups"* - . 

; . ' - ■ /■ ■ ' - ■ . ^ ^ ■ ' ■ ' ■ ■■■ ^ . ■■ ■ 

2.4 Gaus%l^^ Effects and Related Quantities in Causal Studies 

, ■ • • . . ^ - - . ' ^ , . , . . ■■ - , ■ 

Thg^ Qausal effect of t on Y (relative to c) f6r eac|i unit in P is 

gi^en by the diffefrwce , Y- - Y . This is the amount- that has in- 

' ^ . " ■ -'• t c ■ ^ . ■ • ■ ' • ■ ^- . • - - 

creased Cor d'ecreased)'. the .value- of Y (relative to c) on each unit. 
The expected value E^Y^ — Y/;) is the average causal effect of t versus 

■ - ■ t C ' . . - ; ' ^ 

•. •• • 1 - • ■ ■ • 

c on Y in P. ^Since the expected value pf a difference is the difference 
in exppc ted values , i , e . . . ' < . 

- •• : • ■ E(;y^ -t- Y^) ^ E<Y^)'- ECt^) , . - - -JC^:!): : 

.we see that the unconditiorxal means of Y and Y over P have direct 

■ ' : " " "-" ' t - • c. . • . • • ■ 

causal interpretations. 

■ j» 

In a causal s t u dy , whe t he r Bmitrolled or uncontrolled , ; the value 
of Y that Is observed on each unit ,)is Yg , so that when S ^ t , Y^ is 
.observed and when S = c, Y is oifserved. Hence the expected value of 
■Y^ f or the "treatment group" is . the following/ coJiditional exp e c t a t i oni 

treatment group mean - ECY^|s ^ t) . \ (2. 2) 

The mean of Y for thfe "control -group" is' ' 

- control group TOg^n - E(Y^| S =, c) . " (2. 3) 



*In a more general setting the definition of a causal effect. at the 
unit level would not require that the subtraction, Y^ - Y-^, be meaning- 
ful;" This is beyond the scope of = thiS%apBr but ip. discussed in 
Holland and Rubin (1980) : / . f . .. ' ' : 



10. 



In senerai, thare/l^s_ i^^^^^^ ^® . 

■ equal. . Simtlarly for ECY ) ' arid 10; ] S: - , c)..: ^ence, 'in genrnf^. 



nelthar ECY^Is ^ t) nor iC*. | S ^ c) has 



a dirsGt causal intarpreta^ 



;However/ ECY^) and ECy^l^ ^ t) are aa^ays; ralatedf thrmigh .rt 



-b a s ic ~aq uat i oh i 



ICY^X - ECY^I^ - t), PtS ^ :t) + ECY^js - c) PCS = c) 



Similarly, 



ECY^) F KY^jS - c) PCS^ c) H^ ECY^IS ^ t) PCS - t) 



Nota'ifhat equation <2* 4^ involvas^ the avjeraga value, of Y^ ameng :thosa 
unltsvexposa^td ' ^i^arly^ equaWan C2v5h dnvolv^ ; the avera^^ 
. " value of Y among ■. thoaa. units expos ed ' ¥0 t - But j : E(lY^ | S^^ - c) .arid 



C2,5) 



Its 



- dompanidn eCY^|s ^^ t) can naver be directly measured axcept when Y.^ 

and Y ean both be observed oh all units; ' This is the fundamental prot 

lem of -icHusal inference*: In the Appahdi^- we show how experimental 

: randomiEation resdl^as v^is 

. *- - . ... ■■ . " ... 

diffarance between (2.2) and (2*3). 

3. / Lord'^s Examples .-^ 

:Lord ^sas four principal examples ove^ thdStCourse p^his discus'= 

sion in Lord (1967), Lord (1968) , and Lord (1973X/* E^ch example is a 

ficticious research study that aould dascrlbe a real-life investigation. 

Example 1 is usually rafarred^to as Lord's Paradox* The other three. 

examples amplify the Issues that arise there. In this saetlon, we. 

ii]fi£yz0 each ex4mpie In terms of tthe model given in Section 2, " 
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- i' ' ^ . 11. . 

3.1 ' Ex^mpig^l ; - > Two Subpopul^tions Reeeiving One Traatment 

Lor^-s f^ous paradox, is [the centerpleGe of bd and 
Lord Ci973) arid a variant of Is ^TOentionad /brief ly to^^ (1968) 
it is introduced in Lord (1967) with this short paragraph r 
v-n: — A large university is interested in inveatigating - 

the effects on the students, of ;the diet provided , 
; in the university dining. halls and any seK differ- 

encas in these effects. Various types; pf data are 
gathered. In ^ par tlcalarj the weight of _ each ^ 
student at the time ^of hia arrival in'Sept^Eiber 
and his weight the Spllowing Juttev are recprded* ' \ 

^Therf. ^.s ^^lo Other infonftation describing' thisj hy po the t'ical study 

'4 /■■^^\'/ ^ ^ ' ^- ' -'^ \ " ^ -■- ' 

in the three papers;* tut oth'er information is giverij^ describing the.ob"^ 

served ?data veluesV * Nevertheless , ^ from this short description we can 

identify all; off the relevant alenfenta of the model v; Table 1 sunmiariEes 

this identification, ^ - - \. ^ v : \ ; 



* . " Tabl^ l' goes here / ^ } 

j/Thfi question mark (?) in Table 1 is. di^e to the fact that ^Ithopgh 
the dinlAg. hail .diet is clearly the treatments t, whose effect" on = 
stut^entweigrit is sought by tha'study| th^re is^ no^' control diet, - c, 
even hinted at in^ the three pipers* In our model the influence of t 
on Y is always relative' to soflie other condition c* The fact thatj In ^ 
this eKample, Y is vaguely defined and not observed dir&ctly plays a 
crucial role in our analysis of the paradox* .It should be remembered 



TABLE" 1 



I^airfif icatlon of the Eltoents o£ Jtha MQ_deJ_ jji^Jy^ttgJ*^ 1 



Study Design 
■ > P 



The studantSv'at cHa university in the ^^padifilad 
school year J - ' ' u . 

Tha dining hall .diet i ^ ' , 



S :^ ^ t f or Jill units 



Variables Measured 



G * Studartt gfehda^^^l .^ males 2 ^ f e^la) j 
' : The- waigh t of a s tuderit in $ eR tetob , 
y : . The weight of a student in ^ June. 



^8 



ERIC 



f that Y iNpp resenSs^tlVi^ w 



eight; d£v a; stji^ehfi^axposad tp- tfte con^ 



:trdl a±et, r'Sinca no one is e^osed to Cj anyone xanalyEing the data 
J = wil4 he f oread, to make, untest^able "asai^pt^^ the rvalue of in . ' 

©Xder to obtain numerical answf 



y .Tb^r only :. one yerEion=;o X in thip/e^mjia^i it -is meas- 



ur ed i^i^^^ ep tai^ ax ^ p rio r to the V ong a t ■ b f : the ttreittfSBti > heri^a j X Is a 



r CQvariater^-^iriallyy ^^ince all^^^^^^ tudartg ara^ expaaadP-^cp!^^^ 
wa have S £br all atude^ ^ ;C ^ v =^ 



^^Xord\ f-ramea his; paradox in tarms - oifv the analyW^s .pf vtw^^ hypothe- 
ticai statisticians quite dif iarant cprt^^^lusion from tha 

data in; this exart^ieV^'" Pia samplea are all assymad^^^ be large so. that 
/tha foQus is on; the/ interp rotation of the \a,lues^=^^ 

have beemeitimated "with' .high praciaionv' We shall aiiTnmariEe all statia*- 
tical anaiys^a in\t^rma of J tha parametera that^^re estimated. The ; ■ 
affect of the 'dining hall diat ,on a stirdant'a weight is given by thm 
difference Y - Y-. so that the averaee causal ef feet of the di&t on 
'Student waight "is the topectad value of thia difference, Hbwevers ana 
of th^ featurea ;of this "atudy= is an expre;5sed intarest .in any sex 

diffarencas ln:,thase effacta,'' Thus i the average cfuaal effects for 
males and for. ,femalea need to be separately estimdfed, /fhe parameters 
of intereat are the average ^usal affects for males and for fpmale^ s 
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and the ^difference .q£ average causal affeats;, - > - ' 

^In .terms of the ;±ndividual v3ubpopulati©n ave^^ eKpressad ■ 

• either^ as ■ ; ■ ' . ' ^ ^ - ^ ^ r 

or ' ■ ■ ^ ' " / — ^ - - ' . 

- ^ XECY. |G - 1) ^ E(Y~|g^ 2)^^^ (3,4)^ 

■ ■ , . ... ... "■ l' *^=. ' \._ . • V ; . . 

' Equation (3. 4) is" especially useful in this eKample sinca it separates 
the obsarved Y from the unobserved Y , ' ■■ \ 

St a t is tic ian 1 b as es his conclusion about the effact.^pf the diet 
on * the . difference ba tween the distributions ^of Y^ ^nd of X In = each sub« 
population i* a* s /males and fsMlas, In terms of the means of .thaaa 
/ distributions/ the oorrespcyiding parMieters aire ^tha average dlf ferencas 

: - ECY^-- x|g - IK^ i - 1,2. ^ ^ (^5) 
The quantity D> is tha^ mean weight gain in subpopulation/i. The dif-- 



-ferencp of _tha gains is 



D ^ - D^* ; ' (3.6) 



From the" description of the pattern of data values give^ by Lord in 
thi^ example Statistician 1 obsBrv^s that there. .are no differences 
featween the beginning and ehdihg weight distributions for <Bither males 

or females* Thus the in C3*B) are both zero* from this observa- 
tion. Statistician 1 concludes thati. ^ \ r'^ - 



- ^ ^; as . Jar as theae data ara. cbncerneds ther^ - !■ = 

ft ;^is- no a'^idance of any Interasting aflect of - '= , 

e^diet^Cor : of anything elaa) oh^^ , ; . 

' ■ ; > pafettqulai^r^ Is no eyidanee of, ai^: ./ ^ . ^' 

difftoa^tial. .affect on the two axes, ^ince ' ^ 

naith^r groupVshws anyr'systematlc changav^ f ; 
This causal IriJe^noa^is Sao tiua wlthcut making additional ass^tions^ 
The D In (J^S) ar^ nut averago-cauBal effect parameters. , In dfawing 
his conclUslohi-^Statlstlcian 1 is aaklng an assumption abojitj^e numerl- ^ 
cal vaiues of the unobserved, variable Y^. There are ■eve^a^, possible 
assumptions ; ha;. could make to duptiSy his. conclusion. ,'OneSof :gfe^iaplgs^t 
ia to assume that the fasponse to the control\d^^ whatevetrjit might 
be, is given. by. the student's weight in September, T^. ' ■ 

:Undar; this entirely untee table as suction, thaD^^ i are^aqual to 

the .avarage ckusat ef f aota * ^ : . _ _ __ . 

, > In Lord C1968) r-Cord^tSbkas^ a brief referanc^ that is re^ tb tha V 
ksaraptlon C3, 7) , He ^ref ars to critics of Lorf (1967) x^ho\sii^ 
"the obvibus procedure to 'use" is - c^e "gain score'% - X*^ :Wa woul4\ 
interpret such -critics as Attempting to obtain an aatlmate. of -thB causal 
affect of the dining hall dtf^t. on. each rtudent- by making assimptioh (3* 7 
Since assumption (3*7) cannot he tested with tha available data, acceptance 
or criticism of it^must be based on intuition end/or sub jact^matter 



. Statistician 2^ oomput as a covarlanca adjusted differance of the 
twp subpopulatipn- means* TO cpmputlng th^* following 

pwo conditional expectations fi,e. ^ within-group ragrassion tunctions) s 

The maan 5 conditional, weight gain in group i at X is ' 

• . / ^- D.CX) - E(Y^ - X|X, G 1,2, \; - (3.9) 

Tha differehca--in— thase^onSi-tion^^ gains at X. is ; ; . 

: Xi: A ; / : _ . : ; ^ D CX)^ ^:i£CX) P2<X) - ' V ■ :j ^ ; C 3 . 10) 

Foir simpli&ity* Loid assuinas that tha aondltional aKpac^ations in (3,8) 
are both linear and parallel* Thus wa .can write ' : ; 



E(Y Jx, 1) - a^ + bX,' 1 - 1,2. >V " ^ (3.11) 



llances -D^CX) simplifies tb ; * , ^ 

\y_> : ^ D^CX)>^ a^ + (b-l)X, 1.^ 1,2, — ;(J,12) 

and DCX) simplifies to ■ .... _ - . ' ■ : 

J ■ ; • :•• ... ^ClC) -a^- a2. ' V \ (3*l50 

Thus, D(X) ia independent of the value of X, Scatiffticlan ' 2 cprrectly 
interprets D(X) .aW the . average amount mpre that a male (G - 1) will 
weigh in June than will a female (G ^/ 2) of equal initial weighty X, 

AlthQugh. eorrectj this statement aboup^'Iff^ScX bears no direct relevance 
to the differential causal effect of ^W^ining' hall diet on the Juna 

weights of mala and female students,' This is because DCX) in (3*10). 

^ " - ■■•1- . ■ : -: / .- 

is nat directly related to the causal effect parameters A^, and L 

giVen (3*1) and (3*2) * ' \ ' 



r. 
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However^ under a^n untes tab! that Is akin to but " ^ = 

diffarent from (3*7)5 dCX) /equals d and CDnsequently does ^aasura the ' 
diffarantlal eauaal- effect of^lnterlst* To sae this we ganeraliae tha 
assumption X3* 7) to : ^ > ^ . ■ C^ - — 

' ' ' ; ' • . ^ :^ + ex. " : ' : . ca.ii). 

Assumption . (3. 14) asserts that a student's weight in June undar the 
eontrol diet,- Y y is a determ tha value of / 

the student' s weight In Septsmber/," X. ' FurthernQrai the same linear y 
function applies tb all students regardlass^ of . gender , The assumption; 
of Statistician 1 is that of no weight change under the control diet 1 

/f,e,i a ^ O5 0 ^ 1* If Statistician 2 makes the alternatiya assumption 
that s'^-b where- b is the coiamon slope of- the two within-groups regras- 

^ sion^ =linas in (3*11) , then ha may- intarpret DCX) in C3,10) as the ^ 



difference in ckusal' effects * 4 defined in (3 .'2) * We omit the s traigltt- 
fowa^d^ algebra that shows ^this, lliese reaults are sumflarized in 
Table 2: ^' ^ 



Table 2 gpas here 



^ .We wish to emphasize that the assumptions that lead 'to the formulas 
.used by the^two statisticians in Table 2 are not the. only qnas, , nor are 
they the. most general ones. For example ^ Statisticiah 1 could make the 
weaker assun^tion that E<Y^)&^^ 1) ^ eCx|g 4 p instead of (3;7)../Any 
assimiptloh about jr^« must be untes table in this aKample and yet will lead 
to a formula for A* The plausibility of any particular assumption about, 
must be argued from considsrations external to the data. 



2 



is. 

1 



. : ' TABLE 2: V '-.^ ; 

. ' ' . . . _ - ' •- • . ' ^ ■ . ■ . - ■*•.'■.> 

A Sutmnary of Sets ^o£^^^M That 
Lead to the Conclusi'oji of Each Hypothatical 
Statistician in Lord -e Paradox 



Both assume Y ^ d + BX for all unite in P* 



statistician 1 



Statiatician 2 



Tastable 
-Asstmptiohs 



ECY^IXs-^ ^ 1)^ a^^ /+ bX 



Untastabla 
As s ump t J.O ns 



a ^ 0 
6 ^ 1 



Formula for 
causal affecta^ 

1 V 



A. ^ ECY - X G ^ 1) 



>^ = ECY^ - a - bXIG = i) 



Formula for 
differential 
cau&al affect 



A - ECY^lG -a) 
. ^ ECY^jG - 2)^ 
r [ECx|g m 1) 
;^ ECxIg - 2) ] 



EXY^lG ^ 1) " E(Y^|G ^ 2) 
^ blE(x|G ^ D;-* E(X|G - .2)] 



difference in 
mean weight gains 



^ CO variance adjustad mean 
difference in June Weights 
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ati# in many ' cases particular ass umptidns pay reasoriable^.* . 

There ate Btataments ±nlLort^Ci967) and Lo^d (1973) fchat suggest that \ 
Lord would be. willing to accept* the' apstmptlon that juitifiea Statistic 
clan 1 rkther than the orte that ^justifies Statistician 2.- Gur v^ is 
slightly different/ To paraphrase Lordj^^ere i^ no s^^^ pro- 
cedure that ^an be cQuhted on to make;,untestable assimptions that a^ 
correct. In the case of %the diet eKamples neither assumption ^seema 
obviously appropriate,^ ■ \ ^ - " 

Ih sifflmary, we believe that the lollowing views rasblve- Lor-i's ^ 
ParadoK, If both statisticians made only descriptive statements, they 
would both be correct. . Statistician 1 makes the unconditional /descrip- 
tive stafcam&nt that the^ average weight ^galns. for Mies ■and:; females: are 
equal; Statistician 2 makes the conditional (on X) statement that, for- - 
males. and females of equal Sep tenber weighty the males gain more than 
the femalea. In contrast, if the statisticians turned these descriptive 
statements into . causal statementB, neither would- be fcoM^ectvor Incorrect 
Vecause uhtestablevasii^ determine thi correctness qf- the causal - 

statements* These sets of assimptiphs are outlined in Table 2* In a 

■ ' = ' ' - . - - . ' - ^ " " . = ... 

sense then^ Statistician 1 is^ wrong because he% makes a causal statement^ 
without I specifying the assumption-neWed to make it true* Statistician 
2 is more cautious, since he makes only a descriptive stateinent* How-^ 
ever* unless he too laakfes further assumptions* his descriptive statement 
is completely Irrevelant to the campus ^'dietician' s interes^ in t^ 

- - - . ■ - ^ ■ ^ . _ ^ __ , ■ - = - - 

affect of the dining hall diet. . ' v • 
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: This ^K^ple is si'^eii^t thd'be^ (1968) is aiiLriiiua" ; 

tration of a type of iltuatipn in which the analysis of covarianca ia..^ v; 
often applied* Lord gives ^ only the following discussion of EKamplk-2^ 

a group of underprivileged a tudents is to bje v 
- compared with a control .group » on fresh^ grada- 

J. ^ point average Cy) , ^e underprivileged group 

has a considerably lower mean , T 

=: ; - ;than the controi . group* ' However^ ^ the. underprivl-' 

^ TLeged group started with a considarably lower - 

mean aptitude score C^c) , than did the • control- . 
/ - g^Q^p. Xs the observed difference between groups \. \^ 

^ on y attributable to initial differences oh x? 

. ' ' " - - Or shall we conclude that the two groups achieve , - v 
/ = " . differantly even after allpwing for iTnltial j , ' - 

.differences in measured aptitude? \ . 

In attempting to identify the various elemeht a of the mdde^ /. 
.Section 2 for this example we must-^4ecide whether this study is. in- 
tended to be descriptive or causal* This deciaion hingea on the / 
interpretation given to the "contrcl group", "Underprivileged"" refers 
to a vague mixture of socials nutritiohals\^=economic and iedueationml . ^ 
, circumstances and somatimea even to^ racial differences.* In some un-. ' ; 
usual circumstances J such as with, twins separated shortly after births 
it Qan be reasonable to 'consider "underprivileged" as an experimental 
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; mahipiulatiorif sucb^ casasV both the "m^ah a^ " • 

;fi:ashman grade point average Y 'woiild be: stf fected by exposure to this- ' . 
/.experimental manip^ and both would be represent Qur model ; 

by "two verslafnss l,a, ^ X^, X^s Y^p .y^* Although it Is conbeptualiy / 

■ -.possible to regard "cdntro^; and ''imderprivileged" as twd levels pf i 
an experimental manipulation. In practice .it is ofte^ unreasonable to ;^ 
do so since the eKposure essentially begins at birth* Hence .we shall 
interpret this, example simply as a descriptive study in which there 
are two' subpopulations.^ (1. e*^ "underprivileged" and the "control 
■ groups') being* compara^. Table 3 identifies the elements of the model/ 
with &he /interpretation of Example 2 as a descriptive study. 

. . ■ : . ■ ■! V. ^ . Table. 3 goes, hwe ! 

The concomitant variable X defines a subpopulat ion o£ for^ach 
= ^ of its values, i, e. the wbpopulation of P for which X ^ 75. In terms 

of our model j , it is not possible to ask if the value of Y for a unit 
f would be .different ■had__the ^alue: of _X Aat^ unit been ] 

This fact^randers meaningless'the question of whether ^ not an observed 
difference between two grqups on la attributable to differences in „ 
= the values of X\ for the two groups . In order ^ to attribute cause to the 
. values o f a var lab le ( 1 * e * , to ?es t ima t e a caus al / ef f e c t in our mddel) " 
" it is necessary ^for these values to 'indicate the levels ; of a treatment - 

Hence, causal EtfL^ements involving the influence of a concdmltant^ ' , \ 
; ^variable on a dependent variable #re generally not meaningful- However,' 
*there are useful descriptive parameters' that can be espiMted in this - 
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: ^ 22. 

\ Idantlflcatlori of the Elements of the Model In Example 2 - 

Study Dasign . ■ ■ . - • 

' . P s " The freshman claes at the university in a. 

givan year* , . • 

Variables Heasurej, 

G : Underprivileged status (1 ^ undai-privilegedi =^ 
2 ^ Gontrol) , - ' . 

: ..1 . X r Seora on an aptitude test taken prior lto___.; ^ ^^^^ , 
college entrance., 

Y : Freshman grade^^polnt average. ; - - ■ - / 
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^V-'^o-V' V ■ . '- [ , ..■^'^ 23, -■ . . 

type /of V study;. The mean difference between the grade-point j averaga pi 

students in the two ai^pcspulatlQns^wlth the aa^e valua Jb^ Xjis 'glvan^y 

the difference between the two regM - ; ^ . ; 

^ " ' ' ' ! - E : ' ; .(3.15) 

This difference may ^Ve uaef ul for predictive /purposes , -but it cannot 
be given a' causal interpretation in, our iflodel* 



3^ 3 VExample 3 i Contemplating Hfew Treatments ^ ■ i 

'/Xord gives this example in Lord C1968) , His description is as 

follows: „ . * 

'----^----^.y^ --- ^ - - : ; ^ SupposW: an agronomis t is s tudy±n&^ 

' - ■ . ' . ' ' ' ■- ' - . ■ ■ ■ £ ■ ^ " « 

of various varieties of corn. He plants 20 
flower pots with seeds of a "black"':vari.ety 
and 20 more ^ots with seeds of a "white" - 
■ : / variety* For simplicity of illustrations 

suppose that he treats all .40 plants equally 
■ for. several months p after which he finds that; 

! the white variety -has yielded considerably 

- ■ more marketable grain tHan the' black variety* 

V However^ it is a fact. that black variety 
— : plants average only^ 6 f eet high at flowering 

: V time p whereas white variety ^plants average 7 

feet. He now asks the question^ would the 
- ' . Hlack variety produce as much salable grain i'f 

" ^ conditions were adjusted so that it averaged 
\. ^ 7 feet in height. ^tflowa^ing time? 
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Table 4 Idancifiaa the elemants of the model In this 'exa 




. ■ . -. V = Tab la 4 goes here . ' ' 

This axampl^" is li^je— the first ^na in chat only ona laval of the 
experimental manipulation oceurs in the study, ^ However ^ Lord is quite 
clear In this example as to the problems created by not having an ex^ 

plieltly defined alternative eKperimental condition. In factj the. 
question he raises in this axample concerns the choice of t. In his 

words : ■ - . . • . 

In practice 5 the , answer depends on what wa do. ' 
to secure black-'^^t'iety plants avaraging 7 \ 
;-- f^Q^ i^ height; : This eould^ be -dona by dM troy-: :^ 
ing /the shjarter plants j ^ by applying^moM * - 

fertili^eri oif by stretching the plants at 

night while .they are young, or by, other ..means * ^ : 

; The. an6tJ0a/L dgpmd6 on 

^ " Thte: role of the^cdr^ variable in this example -i^ quite v 

different frcan the previous ones . It is eyident" that the measured-^ 
value of X will be affected ^by t since that would be the atated? pur- 
pose pf the treatment , Thus, there are two . versions of X*/ ^"J^^c^ 
and only X is measured in this study. Not only must one make untesV 
able assumptions as to thevalue of Y^, it i^ also necessary to make 
assmptions about ; the value of X^v - xhe parameter bf interest in this 
e^mmple is the average causal effect on yield for the "black" variety 
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, Identif ication ^f tha^^ thm Modal in E3c^p_i_s _3 

Stud^ Design V ■ J • . 

. = P : Corn feaada. •. " " - ; ■ ' 

■ t ? - ^ ■ . ' . " 

c ' : Tha " standard'' treatment applied tsy tha 
agronomist- - . 

S : S c for all units, . ^ ■ 

. . ' •„ - -. ■ : '. ' . . : . ... . 'c-y. ., . •' . . . ...... •.. ■ '^j 

Variables Measured : 

G I Corn variety (1 ^ "black" , 2 "white") - 

. X . t Height at flowering time,' ■-. 

, .; ^ .' Y I Amount of marketable grain prpduce^d- 




/ • : • ■ r ' " ■ E (t- I G M ^_ E CY^i G - ; d > e cy^| g i)V : " : ■ " ( 3 . i6) 

The value of E(V^|:6 « 1) can be coBputad from the data, but the value 
of E(ir-|G ^ 1) is determined by wha,tever untestable assianpt±.ons/wa makei 

A ' ,:.Let (x) ^and u^Cx) be defined by , - ■ t , ' , n? 

- - y. CxO' = ECY^Ig = 1, « x)\ \ - ^ .: 

^ -'^ . ' ^ r * • . I (3.17)- 

, so that p^Cx) is the. ragreselon of Y on X under treatinent t .for the 
"bladk" varlaty, and y^Ck) Is this, regression "lunder treatmant c* To . 
bb t ai n an ana lys ie of CQvari ance' ' ..s Q lu 1 1 on ^ we l^aay \ as s ums_ tha t ;thea e ^ : - 
two regression functions are equal i. i.e. . . \ = . ^ 

V V ^ ' p^C3^ ^ y^Cac) . : - V ■ C3.18) 

Let' us also auppose that this regres.siori is. linear j i*e." . 
• • ' ■ • . •• ■ u (x) = a + b X, . \ - ■ 03-19) 

Assumption (3-18) is untestablej but assiwaption^C3i'19) can b - 
wl th the 'data . ^ We may then Gompute the unknown ! quantity in ( 3 * 16) , 
E(Y^|g = 1), by the formula. . "- , ' - ' ; 

EC^^jG - 1) - ECy^C»p|a 1) ^,a^ 4- b^ E<X^jG^^ 1) *^^ 
Since the mean of'Y o^for G -/ 1 can be eKpresBed^as 

ECY |G^^ 1) ^ ECy (X )|G^ 1) ^ a +-b ECX \g r V), ' <3.2l) 
/ tl^ average increase in yield for the '-black"^; variety la; 

: E(Y— - Y Ig - 1) - b [ECX^Le 1) - E(X |g^ 1)J A (3,22). 

t i c' ^ c ty c* 



which is an "analyals of Govariance^' solution* However, we agree with 
Lord that the plaualbility of /the unteatabla assuittption (3*^.8) depends 
on the .choiee of t. For eKample, it might be^ a plausible assumption i¥ 
"additioni^i fertilizer*' is the new treatment , but '"stretGhing the young: 
.plants, at night" might only^ lengthen them with no corresponding change 



in^'ytpld or^^^ theini:^nd in^^itEar case. (3*18) 4ou^ be ■ 

.apptopjri^te.";- ■^'''■.^■/-* "-J^-. ■ " ^ . ^-^ - ' : ' "v":^'.^^ ■-- 

3* 4 ^Mple r4 i Two Explicit Trisatments - . r- / ^ V ' , ■ v 

, Al t h ough tti e f Irs t t hr a a e^mp les a r e in t end e d t o J i llus tr a t a car—/ 
tain ;pbi^ti and are .not considarad by Lord as indicktlve ;o v 
research studies, ' the /"Mnal exa^las ;£S Lord (IS^Sy^ 

that "the^ paradox /is not just-an amii3ihg statistical Lo^'s 
it^tamant q£ the axaMple- ie as follows, : / = - 

p p , consider .tha problem of evaluating ;faderally ; . 
, : funded spaciai education programs; A group of ^ . . 
disadvantaged; children are pretested in September, 
tfcen enrolled in a special program^ .and finally 
. pos t tested in June* A_ contrQl gro up ^.of children _ . . » 

ara similarly pretested and postteeted but not 
/ , enrolled in tha special program* Since tha most . 

disadvantaged children are selected for tha 
. * Spaciai program, the cotftrol. group. will typi- 

_ [._'_._._ _cally have higher pre t a scores, than the dis^_ _ _ . 

^ > advantaged grioup. ^ 
, This is the first of chase axamples in wHich two levels of an 
experimental manipulation ara axplicitly present. Table 5 identifies 
tha elements of the model in "this example* . 



Table 5 goes he,re 
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vO / ' = ^ 28, 

V"V^ -. ^ ■ ' ./ - " ' tmiE 5 - ^ 

^: . ' ■ -^^ ^ideatlf icatloti of tha Elmants of the Model in la 4 

> "Study Das ign - \ - i - . . • . . 

^ P ^= "^Tha^:;atudents in tha specific schools in the 
^ ^ givan school yaar* . , 

t r'-;Tha spaelal education program* ^ ^ v 

Q : ^Tha standard aducatipnal program* 

S : . Treatment indicator * 

Variables Maaauy ad ; ^ 

. ' '- ' ' . . - ■ . ■ ' . . , ' 

G I Disadvantaged Indicator (1 = 

^ ; 2 ' - control) * ' ' 

. . ' . ^ \ ^ " ■ 

■X % Pretest in Sap teller . - - - ■ 

Y. 1 Posttest^in June; ' . ^ 



34 



ERIC 



; Even^ though two treatmants ara eKpllcitl^^ there' is am- 

biguity as to how they ara asslgnad and what -the relationship batwean 
;S ,and G is,: TOa ramark "Sinee tha most /die advantaged children are / 
selected for the spaeial prbgrmV, might be read as meaning that the 
selection «.bi^a uiiii into a: treatmarit ;grdup la /ta^e ;oh^ thC baii-s of the 
pratast score, with the lower scoring ^children more likely to be ^ 
enrolled in the Special prograta. On the other -hand* the description 
might be interpreted as implying thit-S'^ G indicates;^ 
classification of -^children, into "disadyarttagad" and "control" , not 
^'ter^ned by The^dlff eranMs^^J^^ 
of fundamental^ importance* » , 

Firs t 5 'suppose chafe assignment to^ t or c was* based on the value 
of X5 and that the values of G are jus t .labels determined by theVv 
covariate X. If the regressions of and-Y^ on X ar'e linear and 
parallels then s. as we show, in the AppendiXs the usual coyariance 
adjusted estimatoi^ est Ima tea the causal effect^ E(Y^ = Y^) * 

— Ijti contrast, suppose that S'^ G. and that thfere are two ekis ting 
sub populations indicated by G,.. and that G is not a function of X alone 
Now S and G ara cpmpleteiy confioundedr Sf? that In order to estimate Jth 
effect of t vs. c on Y for. each subpopulatlons wa.must make assumption 
about the values . of Y and Y for the groups exposed to c and t raspec^ 
tively* These assumptions will be untastable and similar to those 
made in EKmple 'I. ' - ■ . 

4 • Pis cuss ion . ^ 

» Wa. believe that Lord touched upon a number of important Issues in 
the. examples that surround his paradox. . The blind use of complicatad 



s ta tls tlicat pioceduree t ^ liiiV tha analysis ■ pi^^apya±±^ae.s ±^ doomed Co "^^ 
lead to absyrd .conclusiona; On tjie/bther: hand, the lanalyais of ^ co^ 

= varianaa is, a- usaful tool th^t can often render an apparently inp 
table problem manageabier We think that the lvalue of the ™del ! ; v 

- described in" SSeifibn one to thin^ carefully about 

tha attribution of cause- Causal statements made in natural language 
are of ten vague and potentially^ misleading* The rQle^o^ . 
is to give precision to natural langiiage statements, and we bfelieve 
that this is an Important aspect of our analysis of Lord's ParadoK* 

^Zi f hill^^ the "appropriate -Wg^^^ to- if es olve^. Eord * a : P^fSddx is ; 

to be absolutely explicit about the untes table assmptlons that peed 
to be made to draw causal inferences. These assimptions all involve 
the responses of units to a treatment to which they are pne^osed and 

f^thereby turn observations abbut data /(I* e. ^ descriptive conclusions) ' 
into' causal inferences * V Wa only 'disagree wit^^^ tone of Lord* e three 
arHcles that suggests the analysis of covarianci cannot tfe trusted 

'except under special e^erimental designs. We feel that burMnodel 
shows. that ±n complex studies In which causal inferences are of 

concern, "there are always both testable and untestable..assumpti^nsr that 
must be made, in ord^r to draw causal conclusions* We believe that it 

.is both scientifically necessary and pragmatically ^helpful to make 
these assumptions explicit.' _ ^ - ' 

The distinction between causal inference and .descriptive inference 
is essential in many; contexts * and this distlnctipn is qlarifted by our 
framework. For examples questions such as" "Is the new diet more effect 



ttvm f oi i^X^ aLre causkl and/ Imply ^^^a edfinparison of an^ . , 

butqoie. for ' the new diet ' with /^n ' outcome for^ 

"Sounding qu$s tibns may not be causal and ^ Invdlye . no at trlb ut ion of , - 
causa. For example s .*"^o gained more under the new dteti males ^ or , . 
females t*' is not a causal qijestionp but a purely descriptive one^^^^^ 
as such, it can be answered without , making the 

f of^Saus al inferences . ■ - pes'ct iptlve ques tiohs dif f er== f rom^caus al ques^ 
tlons in' th^t there is no implied comparison, of the values , of an out^ 
'come variable under different levels pf an eKperiitiental manipulation*" 

As illustrated in the Appendix, -the \calculati^^ 
answfer^ descriptive questions may, in soine cases, be identicaiy to the- ™ 
calculations thMt are required to answer causal questions under speci- 
fic assumptions,- The. scientlfic^^^ practical .interpretations of the 
results of the calculations are j howeyer, drastically different -^or .:^^.^^^ 
descriptive and causal questions'. The AppendiK shows how experimintal 
randomiEation can alleviate the problem of having to make untes table 
assumptions to. draw causal inferences* This. should not ba interpreted 
as meaning that randomlEation is necessary for drawing causal inferences 
In -many cases, appropriate untes tab le-^^^^^^^^ 

by intuitionj theory, or past evidence, in .such cases, we should not 
avoid drawing causal ^inferences and hide behind the cover of unimterpsjt-^- 
ing descriptive statements* Rather we should^make causal statements 
that eKplicate the underlying assumptions and justify them as well as 
possible* ^ ■ = , . ' \ i : . 



^pendixi Randomisation arid Infaranca £6r Causal Eff fects \ ■ ^ ] ^: 

We now shall show how randomization and^^^^r^ can be; - j,: 

brbught Into the modar and how they allow causai inferences to be drawn, 
using standard statistloal methods* ' , . r ^ ^ 

A.l - The Completely Randoml'aed -Experiment ; r ^ ^ - " V 

; Randomisation has a powerful effect and a special place ^in our- 

model* In a complately randomi zed 'study, great effort is made to In-- 
sure that S is statiatically Indepehdent o£ all other variables in the 
In particular S is made to be/ independent of Y^.andY^. Hence 



ECY ) - ECY |S - t) - ECY^IS - c)^ ^ CA-l)- 



we have 



and 



. - ; ECY r:« ECYc S - c)- -;ECY S - t) ,v ^ ^ - - - - : - 

: .C y c . " , c : , , 

The crucial consequence of randomisation in our Tno4fel is that it forces 
- the" equality, of " the .average cauMl effect and the treatmenG^control-'- 
" groijp mean differences . . * ^ 

: \ ECY^ ^ Y^) - ECY^Is - t) - ECY^Is i c). ; CA-3)' 

Av2" C^us air Effects in Subpopulatidns r „^ ^ — _ _ 

When subpppulations have been defined using G, it is natural to 
want to. estimate a causal effect in each subpopulation. By analogy 
with equation. (4 # 3) the average causal effect in subpopulatidn 1 is: 

' ECY^ - Y |G - 1) - ECY^I g - i) ECY Ifi - i) • — (A, 4) / 

Thus, the unconditional means \of Y^ and Y for the units with G. ^ i 
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have 'direct^ causal InterpratationsV ^ How the eKpected values of Y 

for- ttaated and coritrpl units w±to^>G: ,11^^ by* lit analogy with- : 



.C2.;2) and C2;3)v 
t r e a tmen t ; g r oup_ mj 



1 



eQfttrol grgup mean for- G ^ 1 lunits :^ e€Y,^| S ^ dgv G v : \ CA, 6) : 

The qua'ntities In (A. 4) \are\ reJ.atad to the quantities 'in (A. 5) and (A. 6) 
by tha follQwlng equations which are analogous to aquatlens (2.4) 'and 

- : ECY |g - i) - ECY. |sV- t, G^ i> PCS - X , ; 

. ; , . 4- ECY^|s ^ G, G ^ i c|g^ i),- V - 

: ECY jc ^ ±) ^ ECY G ^1) PCS ^ cJg -/O ' \ 

■ - •-- . r + ECY^Is = t, G - i) PfS ^ t |G = i). " . ■ ^ Z- - ■■ - V 

Note that equation (A, 7) involves the ma an ot for units eKposed to 
a with G n i and equation (A.8) involvas the -mean of Y^ for units eK-. 
posed .to t wj.th G^^ i, i ^ ls2. But ECY^ | S > G - i) and V : 

. E(Y^js' ^ t^ G ^ 1) ; can never be directly measured. As with causal ; ^ 
r fef fact s in^ the population *" randomisation plays a special role when- - . 
estimating' causal effects in subpopulations * * , . ' ^ ^ 

.A, 3 Randomisation Within Subpopulations ^ 

\ Suppose that wittiin each subpopulatipn, S is independent of- 
(Y^,'Y^>. This will hold i for aKampres in eoiEpletely randpmized eKparir 
ments and in "randomized block" >eKperiments , where different randomika= 
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tlon- xules might be used M thin :aachTOub.pbpulat^ axaraplej when; . . 

G 1> the probability of being treated ^j^. if x^mTm^m [whrnn G^^ 2j - the 
/ probabjillty of being: t is ;\iS. If- S ±b conditibnally indepltndeht 

^ : 'eCY '|g - i) =^ ECYv[s;-^^t, G^Vi) -^eCy. |s ^ criVq = i) , : - 
arid \\ - ^- /' --'^^^ - - , ... v. ■ ' ■ ! ^ ^ - 

r ECY |g ^ i)"^ ECY js^ G, i) ^ ECY ^ | S . ^ t , GV^ i). : ^ 

Thus randomization within sub pop ulat ions for^cas the Within subpopulati^n 
equaiityj of the aveTage causai\ af feet and the" traatinant^control-group 

"-yean diffarance^ i,e, . ... ^ ' C ■ 



r-r- i) ECYgljs ^.c^;;^ G « i), ; : - ; ■ ! v: 

Al 4 ■ RandomiEatioii" Based on a Covariate ' ^ ^ ' . ^ 

' ; - Suppose the concomitant X i"i^ a Qovariata so tha"t : X.^^ ? ,X^« . 

^ W^eh a covariate: is Qbserved.bef treatment condicione kre salectedp: . 
' it can be used, tp a conditions, .For examples 

^" . .le& iC be a pretest, and suppoae atuderita with low scbrea of^X air©^ as-^ 

^ Signed with high ^ prbbabiHty to take a special ediicatipnal^ program^ 
^ those with middle scorea a^e asalgnad with ^qual pro^ ability to the..^ 
; special and regular programs s . and those with High scores a're assigned . - 
with high prpbabiiity to the regular^ i 
- . ' In auch a situation^ the randomisation is a funct ion of the ob=;. 
- 'served value of and it followa that S 'and : Y. , Y^ are conditionally 
' independent given X, Hence, \ : ^ , , . ^ ^ 
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35. V 



The iffiportanca of vei|uat±ons^ and (A, 10) is; that frbm tha obstrvad 

data (Yg , S X) we -pay estlm thaaa ra^rasaionsi ^ 

- E(y;^|s " t, X) and ECY^|S - c, X^, .. \. . 



From (A* 9) ^nd CA#3L0) it followi that thaaa regrass ions equ 
and E(Y^|x) , raapa^tively . Now aupp.oaa that ECY^ | X) : arid ECY^ I X) ara ; 
^^l±haar,*-say^-.^--^ 



and 



ECY |x)^ ^ ct>^^> S X, ■ . " (A.12) 



Then tha laast squarea ragraaslon of Y^'on^X for the treatxaent group 
units estimates eq'tiation (A. 11), and the least^ squares regression of 
on 'X for .'the contrpl group ' units estimates equation (A. 12) . (Of course, 
there are other ways to estimate these . conditional expectations when 
they are linear and more generally ^ when, th ay are not ( e _ g . , see Rubins 

.1977)). , ■ ■■' " : ' . ; 

• . -■ • . . . ^. ■ ^ . • • . . - ■ ^ " ■ . / . . 

. . . Suppose that: we have estimated ECY | X) and ECY | X) ; how can we 

• .- ■ ■ - •• : . ■ ■ -x:. . . . - - ' ■.: • ■ , . . ^ ■ 

estimate the average causal Tef feet E(Y. - Y ) in P? Let PCX) represent 

^ . ■' l... ' ■ ■ ■ ^ • 

the distribution^ qjf* X in, 1^- ::; Then . ■ 

ECY - Y ) E[ECY -Ix) " ECY | X) ] P(X) , ' CA.vl3) 

- ■ t c X ^ . . e ■ 

That is, ' the average causal effect of t versus c on Y in P Is simply 



the ayeraga - value^ of the difference b etwaen , the condi t ionaL axpec ta— ■ • : . 
tlons of Y : and of Y at whera^iie averaga ovetf X weighted to re= 
flact tha prDportion of unita; at each value; of X* ' If ^ ' V ; " 

. ECY |X) ECY I X) = K for all X, (A, 14) 

then tha causal ef fact of t versus c is, the sameffor all X* and equals 
th^ causal effeot of t versus' c in P. Wheh CA.14) hoidSs the .averaging 
in (A. I3) is .irrevalant* ^Assumption (A. 14) (i*a, J . p^allal" ragrassions) 
when combined with the linearity assmaptions (A. 11) and (A* 12) yields 
the^raodel undarlying tha usual covariance adjusted estima ■ That is* 



if 



and 



than. 



ECY^jxy - + 0 X^ 



ECY X) a - + B X 
c ' c 



Thus 4 the /Standard analysis of covariance estimator is appropriate when ^ 
(a) assigninant.,into treatment gifoup is based on X, and Cb) the t and c 
ragressKRns; of Y~on~X "are linear an^ parallSi;^ RiA^ 

this', case and more^ complicated ones . . : V = ^ 

A. 5 Randomization Based on a Covariate Within Subpopulatlons . 

. Tha arguBaent of Section A* 4 can ba eKtended to cases '^ilbh subpopu= 

' ■ . . . ■ - '.- - . ■ ' 

lations. An exampia of such a study would be an evaluation of the 
affects of a special diet (S ? t) versus .,a normal:*diet (S - c) £or 
males (G ^ 1) and femalas <0^.2), in which the probability of assign-' 
ment to treatment depends on initial waight (X) with diffarant assign-- ^ 
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ment rul6s being used: for males -'and females-^ ( - 
,pqurui^ ••PCS; = X|X, - G ^ 1) ^ [1 + X/15q]~ j;ana 2) = 0 .; . ^ 

[1 + X/120J \. ) * SQch cas^p S is conditionally independant^^ ■ 

^CY^, Y^)- -given -CG, ;x) /-i/ ; • " " ^ " , - ^- ^ y^' '' .-^^^ 

/The entire argument of Section :^A*4 can be -applied /separataly to v 
aach subpopulatlon indicated by. G ;^ Having bbtainad as timat as of the 7 
.causal affect, of t versus c In'aaph subpppulatipns these estimates can 
be averaged (weighted by the relative- fraquency- of the 
to bbcain an estimate for the entire populatipn* Aiternativelys the 
differences-be tween the =subpopulatlori^ estimates ^cah^^be i computed in order 
to estimate the differencial causal effect of t versus c in the two 
Su^populatlons;, V ; ^ . . >^ . ; . . 

: It is important to note that this comparison of the sizes of the 
causal effects relies on the assumption of the ;con^ 

of S; and CY^.5 Y^) ' given CXj G) and involves the compartLson of Y^ and 
:Y^ 5 only one of which can be observed.^ ^n each unit | this assumption, 
has bean called "strongly ignorab la treatment arsignme^^^ in RQsenbaum 
and Rubin (1982) and plays a central role in; causal irifaranceV 



A. 6 Pes Aljtiye Studies , ' -^-^ 

De^riptive s tudias are different from causal studies in that 
there is no eKperimental manipulation involved and therefore there is 
only one vgrsion of Y. The treatment indicator is^ not even defined in^ 
this case. For examplej .suppose G ^ 1 for males p G = 2 for femaleSs 
^ June weight in pounds and X = previous September weight in pounds. 
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pha des cr Ip tive quas tion is j How much rao^||g^ ma las weigh ; in June : 
than 'do females ?;':;^ the ansM 

' -^-^ ■/■/V-^r ^ ■■''V'eCyI G - . l) -^ECyJ g ^ 2) -. ■ I., ^i; f ■ ■ :• 

And th^ descriptive question is, "How much more= waight- have^ m 
gainea&'Om Septeinb.ar to June than hava females?" by 

;. E(y - xIg 1) - ECY xIg = 2) ^ V;^ " - 



e(y|g ^1) ^ e(x|g ^ i) 
eCy|g ^ 1) ^ ECY^a^ 2) 



ecyIq ^ 2):''^ iCx g ;^ 2) 



ECx|g^ 1) - BiXjG ^ 2) V - ^ 
More complicated questions = ban be formulated by conditionirife 9^ ^ . Fo.r^ 




example; "ttow mucK moire do males witlL September weight K w^igh in June ^ 
than-^do 'females with the sama September weights X?"- It :ls= answered by 

- ' • ■ . - / . . ^ - ■ . ■ . ^ 

\ E<Y|;G> 1, Xj ^ E(Y|G ^ 2, X). ^; (A.15) 

If the regressions of Y on X are linear ffnd parallel in the subpopula\» 

' ■ r ' ' ' ' . ■ ' . . -. . ■ ■ ^ 

tions J i* e. ^ 

,ECY|fe ^^ i, XX ^ e X/ i ^lp2, ^ 

: \ - = ! . ^ ^ ^ " , " ' - • ^ * ' " 

than (A, 15) equals a — for all which ia estimated by the standard 

analysis of cbvariance estiinator, ^ It ;is critical to realizaj howavers 
that -the ranalysis of covariance-^ 

purely descriptive question^ and not a causal question* ^ 

? If the regressions of Y on X are not parallel in the subpppulation, 
i*e. / if (A, 15) is not constant for all X 5^ then the answera to such 
descriptive questions ,aa "How much more" do males with September weight 
X weigh in June than do females with^ September weight X?" dep the 
value of X, \ Sometimes, an average answer may be desired, and, then, the ... 
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difference given by (A. 15) will be avaraged over the distribution of: 



X in some standard populacloni say Pi 



^ . V; E IE(y|g 1, X) ^ ECYJg ^ 2, X) ] ECX) CAVlfi) 

Although .(A*i6) looks formally slml to CA^13) , CA.13) . is the answar . 

to a causal question since it Involvea the bomparlsbn of and Y > 
^whareas CA*16) is, ithe answar to a das.eriptive quastion since it involvies .7 

the comparison of the distribution o£Y for, two . dlffarent values of G, 
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